Rigorous and rapid evidence assessment in digital health with the evidence DEFINED framework.

Dozens of frameworks have been proposed to assess evidence for digital health interventions (DHIs), but existing frameworks may not facilitate DHI evidence reviews that meet the needs of stakeholder organizations including payers, health systems, trade organizations, and others. These organizations may benefit from a DHI assessment framework that is both rigorous and rapid. Here we propose a framework to assess Evidence in Digital health for EFfectiveness of INterventions with Evaluative Depth (Evidence DEFINED). Designed for real-world use, the Evidence DEFINED Quick Start Guide may help streamline DHI assessment. A checklist is provided summarizing high-priority evidence considerations in digital health. Evidence-to-recommendation guidelines are proposed, specifying degrees of adoption that may be appropriate for a range of evidence quality levels. Evidence DEFINED differs from prior frameworks in its inclusion of unique elements designed for rigor and speed. Rigor is increased by addressing three gaps in prior frameworks. First, prior frameworks are not adapted adequately to address evidence considerations that are unique to digital health. Second, prior frameworks do not specify evidence quality criteria requiring increased vigilance for DHIs in the current regulatory context. Third, extant frameworks rarely leverage established, robust methodologies that were developed for non-digital interventions. Speed is achieved in the Evidence DEFINED Framework through screening optimization and deprioritization of steps that may have limited value. The primary goals of Evidence DEFINED are to a) facilitate standardized, rapid, rigorous DHI evidence assessment in organizations and b) guide digital health solutions providers who wish to generate evidence that drives DHI adoption.

Advancing Digital Health Innovation in Oncology: Priorities for High-Value Digital Transformation in Cancer Care.

Although health care delivery is becoming increasingly digitized, driven by the pursuit of improved access, equity, efficiency, and effectiveness, progress does not appear to be equally distributed across therapeutic areas. Oncology is renowned for leading innovation in research and in care; digital pathology, digital radiology, real-world data, next-generation sequencing, patient-reported outcomes, and precision approaches driven by complex data and biomarkers are hallmarks of the field. However, remote patient monitoring, decentralized approaches to care and research, "hospital at home," and machine learning techniques have yet to be broadly deployed to improve cancer care. In response, the Digital Medicine Society and Moffitt Cancer Center convened a multistakeholder roundtable discussion to bring together leading experts in cancer care and digital innovation. This viewpoint highlights the findings from these discussions, in which experts agreed that digital innovation is lagging in oncology relative to other therapeutic areas. It reports that this lag is most likely attributed to poor articulation of the challenges in cancer care and research best suited to digital solutions, lack of incentives and support, and missing standardized infrastructure to implement digital innovations. It concludes with suggestions for actions needed to bring the promise of digitization to cancer care to improve lives.

Considerations for Conducting Bring Your Own "Device" (BYOD) Clinical Studies.

Background: Digital health technologies are attracting attention as novel tools for data collection in clinical research. They present alternative methods compared to in-clinic data collection, which often yields snapshots of the participants' physiology, behavior, and function that may be prone to biases and artifacts, e.g., white coat hypertension, and not representative of the data in free-living conditions. Modern digital health technologies equipped with multi-modal sensors combine different data streams to derive comprehensive endpoints that are important to study participants and are clinically meaningful. Used for data collection in clinical trials, they can be deployed as provisioned products where technology is given at study start or in a bring your own "device" (BYOD) manner where participants use their technologies to generate study data. Summary: The BYOD option has the potential to be more user-friendly, allowing participants to use technologies that they are familiar with, ensuring better participant compliance, and potentially reducing the bias that comes with introducing new technologies. However, this approach presents different technical, operational, regulatory, and ethical challenges to study teams. For example, BYOD data can be more heterogeneous, and recruiting historically underrepresented populations with limited access to technology and the internet can be challenging. Despite the rapid increase in digital health technologies for clinical and healthcare research, BYOD use in clinical trials is limited, and regulatory guidance is still evolving. Key Messages: We offer considerations for academic researchers, drug developers, and patient advocacy organizations on the design and deployment of BYOD models in clinical research. These considerations address: (1) early identification and engagement with internal and external stakeholders; (2) study design including informed consent and recruitment strategies; (3) outcome, endpoint, and technology selection; (4) data management including compliance and data monitoring; (5) statistical considerations to meet regulatory requirements. We believe that this article acts as a primer, providing insights into study design and operational requirements to ensure the successful implementation of BYOD clinical studies.

Effect of Electronic Symptom Monitoring on Patient-Reported Outcomes Among Patients With Metastatic Cancer: A Randomized Clinical Trial.

Importance: Electronic systems that facilitate patient-reported outcome (PRO) surveys for patients with cancer may detect symptoms early and prompt clinicians to intervene. Objective: To evaluate whether electronic symptom monitoring during cancer treatment confers benefits on quality-of-life outcomes. Design, Setting, and Participants: Report of secondary outcomes from the PRO-TECT (Alliance AFT-39) cluster randomized trial in 52 US community oncology practices randomized to electronic symptom monitoring with PRO surveys or usual care. Between October 2017 and March 2020, 1191 adults being treated for metastatic cancer were enrolled, with last follow-up on May 17, 2021. Interventions: In the PRO group, participants (n = 593) were asked to complete weekly surveys via an internet-based or automated telephone system for up to 1 year. Severe or worsening symptoms triggered care team alerts. The control group (n = 598) received usual care. Main Outcomes and Measures: The 3 prespecified secondary outcomes were physical function, symptom control, and health-related quality of life (HRQOL) at 3 months, measured by the European Organisation for Research and Treatment of Cancer Quality of Life Questionnaire (QLQ-C30; range, 0-100 points; minimum clinically important difference [MCID], 2-7 for physical function; no MCID defined for symptom control or HRQOL). Results on the primary outcome, overall survival, are not yet available. Results: Among 52 practices, 1191 patients were included (mean age, 62.2 years; 694 [58.3%] women); 1066 (89.5%) completed 3-month follow-up. Compared with usual care, mean changes on the QLQ-C30 from baseline to 3 months were significantly improved in the PRO group for physical function (PRO, from 74.27 to 75.81 points; control, from 73.54 to 72.61 points; mean difference, 2.47 [95% CI, 0.41-4.53]; P = .02), symptom control (PRO, from 77.67 to 80.03 points; control, from 76.75 to 76.55 points; mean difference, 2.56 [95% CI, 0.95-4.17]; P = .002), and HRQOL (PRO, from 78.11 to 80.03 points; control, from 77.00 to 76.50 points; mean difference, 2.43 [95% CI, 0.90-3.96]; P = .002). Patients in the PRO group had significantly greater odds of experiencing clinically meaningful benefits vs usual care for physical function (7.7% more with improvements of ≥5 points and 6.1% fewer with worsening of ≥5 points; odds ratio [OR], 1.35 [95% CI, 1.08-1.70]; P = .009), symptom control (8.6% and 7.5%, respectively; OR, 1.50 [95% CI, 1.15-1.95]; P = .003), and HRQOL (8.5% and 4.9%, respectively; OR, 1.41 [95% CI, 1.10-1.81]; P = .006). Conclusions and Relevance: In this report of secondary outcomes from a randomized clinical trial of adults receiving cancer treatment, use of weekly electronic PRO surveys to monitor symptoms, compared with usual care, resulted in statistically significant improvements in physical function, symptom control, and HRQOL at 3 months, with mean improvements of approximately 2.5 points on a 0- to 100-point scale. These findings should be interpreted provisionally pending results of the primary outcome of overall survival. Trial Registration: ClinicalTrials.gov Identifier: NCT03249090.

Investigator Experiences Using Mobile Technologies in Clinical Research: Qualitative Descriptive Study.

BACKGROUND: The successful adoption of mobile technology for use in clinical trials relies on positive reception from key stakeholders, including clinical investigators; however, little information is known about the perspectives of investigators using mobile technologies in clinical trials. OBJECTIVE: The aim of this study was to seek investigators' insights on the advantages and challenges of mobile clinical trials (MCTs); site-level budgetary, training, and other support needs necessary to adequately prepare for and implement MCTs; and the advantages and disadvantages for trial participants using mobile technologies in clinical trials. METHODS: Using a qualitative descriptive study design, we conducted in-depth interviews with investigators involved in the conduct of MCTs. Data were analyzed using applied thematic analysis. RESULTS: We interviewed 12 investigators who represented a wide variety of clinical specialties and reported using a wide range of mobile technologies. Investigators most commonly cited 3 advantages of MCTs over traditional clinical trials: more streamlined study operations, remote data capture, and improvement in the quality of studies and data collected. Investigators also reported that MCTs can be designed around the convenience of trial participants, and individuals may be more willing to participate in MCTs because they can take part from their homes. In addition, investigators recognized that MCTs can also involve additional burden for participants and described that operational challenges, technology adoption barriers, uncertainties about data quality, and time burden made MCTs more challenging than traditional clinical trials. Investigators stressed that additional training and dedicated staff effort may be needed to select a particular technology for use in a trial, helping trial participants learn and use the technology, and for staff troubleshooting the technology. Investigators also expressed that sharing data collected in real time with investigators and trial participants is an important aspect of MCTs that warrants consideration and potentially additional training and education. CONCLUSIONS: Investigator perspectives can inform the use of mobile technologies in future clinical trials by proactively identifying and addressing potential challenges.

Determining Minimum Wear Time for Mobile Sensor Technology.

Part 1 in the DIA Study Endpoint Community Working Group on Mobile Sensor Technology (MST) series addresses considerations that may be useful when determining the minimum wear time associated with mobile sensor use to ensure reliable estimation of the clinical endpoint under consideration. What constitutes a minimum valid data set is a dilemma facing those using MSTs in clinical studies. If this alignment does not occur, the integrity of the data collected and conclusions drawn from these data may be in incorrect. While study participants should consent to engage with MSTs as defined in a protocol, participant behavior or technology lapses may result in capturing incomplete data. Drawing from the literature, we review what constitutes a minimum data set, the risks associated with missing data, alignment with the clinical endpoint(s) and goals of a study, as well as managing patient burden.

Learning from patient and site perspectives to develop better digital health trials: Recommendations from the Clinical Trials Transformation Initiative.

In order to harness the potential of digital health technologies to enhance the quality of clinical research, it is critical to first understand how to engage patients and research sites when planning and conducting digital health trials. To pave the way for the more effective use of digital health technologies in trials, the Clinical Trials Transformation Initiative has developed the first comprehensive, evidence-based set of recommendations for incorporating patient and site perspectives in digital health trials. While directed primarily at sponsors, these recommendations are expected to be valuable for all stakeholders including investigators.

Performance Measures Based on How Adults With Cancer Feel and Function: Stakeholder Recommendations and Feasibility Testing in Six Cancer Centers.

PURPOSE: Patient-reported outcome measures (PROMs) that assess how patients feel and function have potential for evaluating quality of care. Stakeholder recommendations for PRO-based performance measures (PMs) were elicited, and feasibility testing was conducted at six cancer centers. METHODS: Interviews were conducted with 124 stakeholders to determine priority symptoms and risk adjustment variables for PRO-PMs and perceived acceptability. Stakeholders included patients and advocates, caregivers, clinicians, administrators, and thought leaders. Feasibility testing was conducted in six cancer centers. Patients completed PROMs at home 5-15 days into a chemotherapy cycle. Feasibility was operationalized as ≥ 75% completed PROMs and ≥ 75% patient acceptability. RESULTS: Stakeholder priority PRO-PMs for systemic therapy were GI symptoms (diarrhea, constipation, nausea, vomiting), depression/anxiety, pain, insomnia, fatigue, dyspnea, physical function, and neuropathy. Recommended risk adjusters included demographics, insurance type, cancer type, comorbidities, emetic risk, and difficulty paying bills. In feasibility testing, 653 patients enrolled (approximately 110 per site), and 607 (93%) completed PROMs, which indicated high feasibility for home collection. The majority of patients (470 of 607; 77%) completed PROMs without a reminder call, and 137 (23%) of 607 completed them after a reminder call. Most patients (72%) completed PROMs through web, 17% paper, or 2% interactive voice response (automated call that verbally asked patient questions). For acceptability, > 95% of patients found PROM items to be easy to understand and complete. CONCLUSION: Clinicians, patients, and other stakeholders agree that PMs that are based on how patients feel and function would be an important addition to quality measurement. This study also shows that PRO-PMs can be feasibly captured at home during systemic therapy and are acceptable to patients. PRO-PMs may add value to the portfolio of PMs as oncology transitions from fee-for-service payment models to performance-based care that emphasizes outcome measures.

Patient preferences for using mobile technologies in clinical trials.

The use of mobile technologies to collect participant data in clinical trials offers a number of scientific and logistical advantages. However, little is known about potential research participant preferences about how to incorporate mobile technologies into the design and conduct of a trial. Using a web-based survey which described hypothetical mobile clinical trial and traditional clinical trial scenarios, we explored patients' perceptions of and willingness to participate in mobile and traditional clinical trials, their preferred trial procedures related to the use of mobile technologies, and the preferred attributes of mobile technologies. The majority of survey respondents reported that they would prefer participating in a clinical trial that used mobile technology than a traditional trial that relied on standard in-clinic assessments. They expressed that mobile clinical trials offered greater convenience, a reduction of in-person clinic visits, and greater data collection accuracy. Respondents also reported preferences for the frequency of in-clinic visits during mobile clinical trials, device training and troubleshooting, data privacy and confidentiality, the location of data storage, and user access to data collected by the trial device. As research participants become more involved in capturing their own data to inform trial endpoints, their user-preferences of mobile technology, such as those described here, should be considered in the design and conduct of mobile clinical trials.

Personalized Medicine in the Oncology Clinic: Implementation and Outcomes of the Johns Hopkins Molecular Tumor Board.

PURPOSE: Tumor genomic profiling for personalized oncology therapy is being widely applied in clinical practice even as it is being evaluated more formally in clinical trials. Given the complexities of genomic data and its application to clinical use, molecular tumor boards with diverse expertise can provide guidance to oncologists and patients seeking to implement personalized genetically targeted therapy in practice. METHODS: A multidisciplinary molecular tumor board reviewed tumor molecular profiling reports from consecutive referrals at the Sidney Kimmel Comprehensive Cancer Center at Johns Hopkins over a 3-year period. The tumor board weighed evidence for actionability of genomic alterations identified by molecular profiling and provided recommendations including US Food and Drug Administration-approved drug therapy, clinical trials of matched targeted therapy, off-label use of such therapy, and additional tumor or germline genetic testing. RESULTS: One hundred fifty-five patients were reviewed. Actionable genomic alterations were identified in 132 patients (85%). Off-label therapies were recommended in 37 patients (24%). Eleven patients were treated off-label, and 13 patients were enrolled onto clinical trials of matched targeted therapies. Median progression-free survival of patients treated with matched therapies was 5 months (95% CI, 2.9 months to not reached), and the progression-free survival probability at 6 months was 43%(95% CI, 26% to 71%). Lack of locally available clinical trials was the major limitation on clinical actionability of tumor profiling reports. CONCLUSION: The molecular tumor board recommended off-label targeted therapies for a quarter of all patients reviewed. Outcomes were heterogeneous, although 43% of patients receiving genomically matched therapy derived clinical benefit lasting at least 6 months. Until more data become available from precision oncology trials, molecular tumor boards can help guide appropriate use of tumor molecular testing to direct therapy.

Patient-Reported Outcomes in Cancer Drug Development and US Regulatory Review: Perspectives From Industry, the Food and Drug Administration, and the Patient.

Data reported directly by patients about how they feel and function are rarely included in oncology drug labeling in the United States, in contrast to Europe and to nononcology labeling in the United States, where this practice is more common. Multiple barriers exist, including challenges unique to oncology trials, and industry's concerns regarding cost, logistical complexities, and the Food and Drug Administration's (FDA's) rigorous application of its 2009 guidance on the use of patient-reported outcome (PRO) measures. A panel consisting of representatives of industry, FDA, the PRO Consortium, clinicians, and patients was assembled at a 2014 workshop cosponsored by FDA to identify practical recommendations for overcoming these barriers. Key recommendations included increasing proactive encouragement by FDA to clinical trial sponsors for including PROs in drug development programs; provision of comprehensive PRO plans by sponsors to FDA early in drug development; promotion of an oncology-specific PRO research agenda; development of an approach to existing ("legacy") PRO measures, when appropriate (focused initially on symptoms and functional status); and increased FDA and industry training in PRO methodology. FDA has begun implementing several of these recommendations.

An accelerated pathway for targeted cancer therapies.

A well-defined pathway for the accelerated development and approval of targeted cancer therapies and companion diagnostics would reduce uncertainty, improve efficiency in development and provide an effective incentive for developers.

American Society of Clinical Oncology executive summary of the clinical practice guideline update on the role of bone-modifying agents in metastatic breast cancer.

PURPOSE: To update the recommendations on the role of bone-modifying agents in the prevention and treatment of skeletal-related events (SREs) for patients with metastatic breast cancer with bone metastases. METHODS: A literature search using MEDLINE and the Cochrane Collaboration Library identified relevant studies published between January 2003 and November 2010. The primary outcomes of interest were SREs and time to SRE. Secondary outcomes included adverse events and pain. An Update Committee reviewed the literature and re-evaluated previous recommendations. RESULTS: Recommendations were modified to include a new agent. A recommendation regarding osteonecrosis of the jaw was added. RECOMMENDATIONS: Bone-modifying agent therapy is only recommended for patients with breast cancer with evidence of bone metastases; denosumab 120 mg subcutaneously every 4 weeks, intravenous pamidronate 90 mg over no less than 2 hours, or zoledronic acid 4 mg over no less than 15 minutes every 3 to 4 weeks is recommended. There is insufficient evidence to demonstrate greater efficacy of one bone-modifying agent over another. In patients with a calculated serum creatinine clearance of more than 60 mg/min, no change in dosage, infusion time, or interval of bisphosphonate administration is required. Serum creatinine should be monitored before each dose. All patients should receive a dental examination and appropriate preventive dentistry before bone-modifying agent therapy and maintain optimal oral health. Current standards of care for cancer bone pain management should be applied at the onset of pain, in concert with the initiation of bone-modifying agent therapy. The use of biochemical markers to monitor bone-modifying agent use is not recommended.

Stakeholder perspectives on implementing the National Cancer Institute's patient-reported outcomes version of the Common Terminology Criteria for Adverse Events (PRO-CTCAE).

The National Cancer Institute (NCI) is developing a patient-reported version of its Common Terminology Criteria for Adverse Events, called the "PRO-CTCAE." The PRO-CTCAE consists of a library of patient-reported items which can be administered in clinical trials to directly capture the patient experience of adverse events during cancer treatment, as well as a software platform for administering these items via computer or telephone. In order to better understand the impressions of stakeholders involved in cancer clinical research about the potential value of the PRO-CTCAE approach to capturing adverse event information in clinical research, as well as their perspectives about barriers and strategies for implementing the PRO-CTCAE in NCI-sponsored cancer trials, a survey was conducted. A survey including structured and open-ended questions was developed to elicit perceptions about the use of patient-reported outcomes (PROs) for adverse event reporting, and to explore logistical considerations for implementing the PRO-CTCAE in cancer trials. The survey was distributed electronically and by paper to a convenience sample of leadership and committee members in the NCI's cooperative group network, including principal investigators, clinical investigators, research nurses, data managers, patient advocates, and representatives of the NCI and Food and Drug Administration. Between October, 2008 through February, 2009, 727 surveys were collected. Most respondents (93%) agreed that patient reporting of adverse symptoms would be useful for improving understanding of the patient experience with treatment in cancer trials, and 88%, 80%, and 76%, respectively, endorsed that administration of PRO-CTCAE items in clinical trials would improve the completeness, accuracy, and efficiency of symptom data collection. More than three fourths believed that patient reports would be useful for informing treatment dose modifications and towards FDA regulatory evaluation of drugs. Eighty-eight percent felt that patients in clinical trials would be willing to self-report adverse symptoms at clinic visits via computer, and 68% felt patients would self-report weekly from home via the internet or an automated telephone system. Lack of computers and limited space and personnel were seen as potential barriers to in-clinic self-reporting, but these were judged to be surmountable with adequate funding. The PRO-CTCAE items and software are viewed by a majority of survey respondents as a means to improve adverse event data quality and comprehensiveness, enhance clinical decision-making, and foster patient-clinician communication. Research is ongoing to assess the measurement properties and feasibility of implementing this measure in cancer clinical trials.